The industry around Data Science is continuously growing as new potential is identified in the data that organizations collect and maintain. Historically, Data Scientists have explored new tools and techniques for mining value from data on an experimental basis. Only after experimental proof of concept in the value of the data will organizations typically invest in the restructuring and support of the new tools and techniques. Unfortunately, because re-structuring or amending a functional data ecosystem is daunting, often the newly developed Data Science workload is instead hosted by an internal Information Technology department in a fragmented, customized, and tacked-on manner, for example through a dedicated server. These specialized solutions and dedicated resources increase technical costs and reduce opportunities for resource sharing and algorithm reuse.
The inventors recognized the need to build a flexible Data Science development and distribution mechanism into the backbone of an organization's data solutions architecture to increase analytics experimentation and streamline the experimentation process. The need for flexibility includes flexible support for software dependencies across disparate audiences which historically lead to provisioning of specially configured and dedicated hardware.